Skip to content

Prevent CI from running OOM when oversubscribing GPUs#629

Merged
abouteiller merged 2 commits intoICLDisco:masterfrom
abouteiller:hotfix/ci-gpu-oom
Feb 3, 2024
Merged

Prevent CI from running OOM when oversubscribing GPUs#629
abouteiller merged 2 commits intoICLDisco:masterfrom
abouteiller:hotfix/ci-gpu-oom

Conversation

@abouteiller
Copy link
Contributor

@abouteiller abouteiller commented Jan 31, 2024

No description provided.

@abouteiller abouteiller mentioned this pull request Jan 31, 2024
Open
@abouteiller
Copy link
Contributor Author

abouteiller commented Feb 1, 2024

Allocating 79855878144 bytes of memory on the GPU device failed
that's 79GB, wat?

@abouteiller abouteiller requested a review from therault February 1, 2024 21:12
@abouteiller
Copy link
Contributor Author

abouteiller commented Feb 1, 2024 •
edited
Loading

Even if we want to pursue #615, there is no drawbacks to merging this.

I created #630 to track the more serious defect identified in the CI runtest.

@abouteiller abouteiller marked this pull request as ready for review February 1, 2024 21:14
@bosilca
Copy link
Contributor

bosilca commented Feb 1, 2024

not not of the not not. What was that supposed to mean ?

@abouteiller abouteiller merged commit b304378 into ICLDisco:master Feb 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bosilca bosilca bosilca approved these changes

@therault therault Awaiting requested review from therault

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@abouteiller @bosilca